Systems and methods for remote evaluation of craft skills

ABSTRACT

System, methods, and devices for measuring the performance of a component of a communications network recently installed by a technician using proactive network maintenance parameters, and further assessing the installation performed by the technician.

FIELD OF INVENTION

The present invention relates to proactive network maintenancetechnologies.

BACKGROUND

Craft skills in the telecommunications workforce are critical to theproper operation of telecommunications networks. The performance of suchnetworks often depend heavily on how well connectors, patches, devices,and other physical entities are attached to the wireline infrastructure,and also on how provisioning and other modifications to, and maintenanceof the actual wireline and wireless infrastructure are performed bytechnicians, subscribers, or even robotic devices.

Evaluation of craft skills, especially during training of newtechnicians, has heretofore been a manual exercise, with instructors orothers experienced in the craft visually evaluating the craft of astudent to determine if the student has performed the task properly andin a manner that optimizes network performance. Instructors mustvisually observe the connectors or generally the craft implemented by astudent and look for signs of poor workmanship. In some cases, theinstructor is able to test the craft of the student using expensive testequipment that evaluates the integrity of the craft handiwork. Howevermany instruction facilities lack this equipment due to cost, andcommunity colleges and other adult education facilities likewise oftenlack such equipment for comprehensively evaluating the integrity ofcraft handiwork.

There is thus a need for remote evaluation for all of the above caseswith a low cost solution, and also in conjunction with distributedtraining centers and online, on-demand training and certificationevaluation programs and facilities. Further, automated remote evaluationof craft skills would also permit evaluation of consumer craft wheninstalling their own home network wiring and systems.

What makes this possible is a new technology for proactive networkmaintenance (PNM) in cable networks which allows automated detection ofplant impairment such as breaks, short circuits, or any impedancemismatch which causes a variation in the equalization response of thenetwork. PNM allows remote personnel and equipment to evaluate theintegrity of the wireline infrastructure in cable networks, and todetect any changes in that network when impairments happen, or when newwires are installed in the network.

In none of the prior art for craft evaluation, especially duringtraining of new technicians, does the approach cover the use ofproactive network maintenance technology for remote evaluation oftelecommunications craft handiwork, nor the use of video or photographyor any combination thereof and an automatic pattern recognition systemwhich is trained by proactive network maintenance technology based onprevious patterns that have been catalogued, or the use of cloud-basedor crowd-sourced evaluation of remote craft using video, photography,including any of the above, along with proactive network maintenancetechnology or other network operations procedures, policies, andtechnologies, to accomplish the remote evaluation of craft skills.

PNM as defined by CableLabs is a network management technique fordiagnosing physical layer network issues. PNM features identifyperformance issues by obtaining data from the following:

Cable Modem (CM)

Cable Modem Termination System (CMTS)

Both the CM and CMTS collaboratively

Examples of these features include:

-   -   1) Spectrum analyzer functions—in the CM, full-band spectrum,        NPR notching and in the CMTS, triggered spectrum;    -   2) Vector signal analyzer functions—in the CM, upstream        pre-equalization coefficients, DS channel estimate,        constellation display, and received MER vs. OFDM subcarriers and        in the CMTS, US equalizer coefficients;    -   3) Network analyzer functions—in the CM, DS symbol capture and        in the CMTS, DS symbol capture and US active/quiet probe        capture; and    -   4) Other test points—in the CM, FEC statistics and histograms        (very useful for non-linear impairments like laser clipping) and        in the CMTS, impulse noise statistics, FEC statistics and        histograms.

FIG. 8 diagrammatically shows such features and test points in CM andCMTS supporting PNM.

SUMMARY

System, methods, and devices for measuring the performance of acomponent of a communications network recently installed by a technicianusing proactive network maintenance parameters, and further assessingthe installation performed by the technician.

BRIEF DESCRIPTION OF THE DRAWING(S)

The present invention will be described with reference to theaccompanying figures, where like numerals illustrate like elements.

FIG. 1 shows a system having a PNM for automatic, remote or bothautomatic and remote evaluation of radio frequency (RF) craft skills.

FIG. 2 shows a multi-station setup version of the stand-alone setup inFIG. 1.

FIG. 3 shows an embodiment using a remote CMTS and minimal equipment ina training facility.

FIG. 4 shows the architecture of a stand-alone setup having additionalsensors and stimulators for characterizing the PNM-based craftevaluation.

FIG. 5 shows the architecture of a system that allows a plethora oftraining facilities to be tied in to a centralized network operationscenter and a centralized remote evaluation function.

FIG. 6 shows the details of sub-elements within the remote evaluationelement.

FIG. 7 shows the architecture of a stand-alone system with PNM-basedevaluation of craft skills in non-RF network.

FIG. 8 shows a diagram of test points in CM and CMTS supporting PNM fromPhysical Layer Specification: CM-SP-PHYv3.1-I10-170111 dated Jan. 11,2017 from Cable Television Laboratories, Inc.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S)

The present invention is directed to systems and methods for remoteevaluation of craft skills using proactive network maintenance (PNM)technology as implemented in DOCSIS cable modem technology. The conceptinvolves determining how well technicians are performing craftactivities such as connectorizing coaxial cables on the drop line,grounding block, or within the home, but also for plant technicians whoare connectorizing coaxial hardline in the outside plant. By using PNMtechnology, it is possible not only to detect flaws in craft skills, butalso to catalog patterns of common craft issues during student trainingin craft skills that can then be used for troubleshooting problems inthe actual network. In addition, PNM technology supports the automaticdetection and pattern cataloging of the following common networkimpairments: RF notches, microreflections, roll-off, LTE and radioingress, adjacency, filters, and resonant parking. Such patterns can beadded to existing databases used for automatically detecting craftissues or other plant impairments in actual networks.

PNM-based remote evaluation of craft skills can be used to evaluate theresults of craft activities by students as part of training, as part ofremote certification of craft skills, whenever new equipment is used ornew procedures are established, and finally as periodic qualitycheck-ups for technicians. PNM-based remote evaluation of craft skillscan also serve as a mechanism for supervisors and managers to assess thequality of workmanship of individuals and teams during performanceevaluations.

In various aspects of the present invention there is provided a systemfor PNM-based evaluation of radio frequency (RF) craft skills using 1) astand-alone setup with a single training station, 2) a stand-alone setupwith multiple training stations, 3) a remote cable modem terminationsystem (CMTS) setup, in which the local cable operator's CMTS andassociated modulators and access network is used rather than astand-alone system in the training facility, 4) a stand-alone or remoteCMTS system that uses additional sensors such as cameras and communityantenna television (CATV) RF measurement equipment, and additionalsimulators such as RF signal generators, noise generators and othersignal generators that reproduce typical interference signals that caningress into the cable network, 5) a larger scale remote evaluationarchitecture involving multiple training facilities and access networks,a network operations center, and a remote evaluation element, all ofwhich are connected to each other via a packet-switched network, and 6)a remote evaluation element which includes a cloud-based evaluationfunction or a remote evaluation processor comprising a pattern database,pattern recognition, and an evaluator engine, or any combination ofcloud-based evaluation and an evaluation processor.

In another aspect of the present invention, there is provided a systemfor remoting evaluating optical craft skills using a similar setup,except that an optical fiber to the premises (FTTP) system is embeddedwithin an RF-based remote evaluation system so that optical craft skillscan also be evaluated.

Further features and advantages of the invention, as well as thestructure and operation of various embodiments of the invention, aredescribed in detail below with reference to the accompanying drawings.It is noted that the invention is not limited to the specificembodiments described herein. Such embodiments are presented herein forillustrative purposes only. Additional embodiments will be apparent topersons skilled in the relevant art(s) based on the teachings containedherein.

FIG. 1 illustrates the basic setup of a system that uses PNM forautomatic or remote or both automatic and remote evaluation of radiofrequency (RF) craft skills. A training facility 101 contains allnecessary equipment in this case for automatic evaluation of craftskills by using a stand-alone cable modem system in which a cable modem102 is initially connected through a calibrated and pre-characterizednetwork interface 103, which may be as simple as a piece of coaxialcable that is at least 2 feet long and more typically 20 feet long thathas been connectorized at the highest level of craft possible and servesas a reference for PNM-based characterization of the network RFintegrity. The ideal length of cable used in block 103 can be determinedby those skilled in the art and in using PNM technology, and will dependon the capabilities of the CMTS 105 used in the setup, however a simpleapproach for determining the optimum length covered by the presentinvention would be to create several different lengths, testing initialstudents with these lengths and identifying the lengths that mostobviously highlight the results of student flaws in craftsmanship usingthe PNM technology and the capabilities of the CMTS 105. The resultinglibrary of flaws vs. lengths of connections can also be used to developa catalog of impairments, as described later in this document, that notonly assist instructors in teaching and evaluating students' craftskills, but also network operations center (NOC) personnel introubleshooting and identifying sources and types of impairments seen inactual networks, and in particular what the appearance of impairmentswith different lengths of network interface cables 103 is and how theydiffer from each other.

In the case of the most modern cable modem termination equipment such asthe Converged Cable Access Platform (CCAP), a single system contains allof necessary elements to provide a fully capable cable modem signal tothe cable modem 102.

This calibrated network 103 is connected either directly to the cablemodem termination system (CMTS) 105, or more typically to another fixed,calibrated network 104 that represents a model of the hybrid fiber-coaxnetwork in the training facility, which is then connected to the CMTS105. Finally, IP devices 106 and 107 are connected to the CMTS 105 andcable modem 102 respectively, so that communications can be monitored,the ability of the network to transport data can be verified andbaselined, and results can be displayed to instructors and students whoare being trained in craft skills. Example IP devices 106 and 107include, but are not limited to desktop and laptop computers, tabletcomputers, and smart phones.

After a student performs a basic craft skill such as puttingF-connectors on a test piece of coaxial cable drop line that is the samelength as the calibrated piece 103, this test cable is then used toreplace the calibrated cable in block 103 of FIG. 1, and PNM is used todetermine the changes in the network RF integrity that result from thestudent's cable. Typical PNM measurements include in-channel frequencyresponse, group and phase delay variation, and most generally, theelectronic impulse response of the network with the new cable inserted.Using procedures well-known to practitioners of PNM technology, the PNMtechnology is used to identify any flaws in the student's craft, whetherthey are located on one or both ends of the cable, and how it impairsthe network, e.g. as a short circuit, an open circuit, or merely animpedance mismatch.

This method can be applied to evaluating craft skills for either coaxialdrop line, or also coaxial hardline using stinger-type connectorization,which can be even more challenging to learn proper craft skills thandrop lines. Additionally, if the PNM technology supports interpolationor DOCSIS 3.1 or both, it is possible to use shorter lengths of coaxialcable for test connections.

The fixed, calibrated network interface 104 can take on several forms inthe present invention, some of which include, but are not limited to: 1)a single piece of coaxial cable; 2) a splitter with multiple lengths ofcables coming from each output and proper termination on all but the onein use by the student for testing their test network interface 103; or3) a splitter with one output being a reference, high integrity cablenetwork interface and the other being a network interface with a known,characterized impairment that serves as a reflection for the impairmentsfrom the student's test network interface 103 for ease of PNM-basedcharacterization and identification of flaws in craftsmanship.

This method can also be used not just for spot testing of student craftskills, but also for trend analysis, both for a single student, trackinghis or her progress from initial craft capabilities to final testing andcertification of their craft capabilities, but also for ensembles ofstudents to determine the average, below average, and above averagecraft capabilities that are possible at this stage of student training.This can then be compared to PNM-based evaluation of craft skills oftechnicians with years of experience and thus used to set long termtargets for craft levels and plan training programs for achieving theselonger term levels at the earliest possible time in the students'careers. The same setup can also be used as a periodic quality checkupfor existing technicians, to ensure that craft skills are still atrequired levels for network key performance indicators (KPIs) that areused to grade all technicians. Further, a series of connectors ofvarying degrees of integrity could be created to teach both students,instructors and NOC personnel the relationship between such flaws andcustomer experience by testing services on the IP device 107.Correlating known impairments to service impacts as described in thepresent invention then serve to develop a new or enhanced grading scalefor technicians in the field based on their craft work.

Another example use of the present invention is for characterizing thequality of different connectors themselves, and to test differentmanufacturers' connectors when they have been applied with what isdetermined from the present invention to be the best possible craft, andsee if there are performance advantages of using one connector vs.another. Characterizing multiple workers' example craft on theconnectors across different connector types would permit a histogram ofPNM-based craft evaluation KPIs such as number and depth ofmicro-reflections to be compared across types of connectors. Connectorsthat give the best average and lowest standard deviation in KPIs couldoffset any additional cost of connectors via the reduced customer issueswith such connectors being used. These results could be compared andcontrasted with other variables such as the ease of connectorizing themon, immunity to elements (water, humidity, temperature changes) and ofcourse cost.

One method to enhance the performance of the system for rapid andautomatic evaluation of student craft skills is to subtract thecalibrated network interface 103 initially used from the PNM resultsusing the test network interface in 103 so that only the channelresponse difference from the student's craft relative to an ideal craftis displayed to instructors or to NOC personnel. Additionally, sincethere may be some micro-reflections that come from an ideal calibratednetwork interface 103 over time as it is used in student training, theresponse of the calibrated network interface 103 should be re-capturedand compared to the original reference to ensure a new calibratedreference network interface is not needed.

FIG. 2 shows a multi-station setup version of the stand-alone setup inFIG. 1. The ways the multi-station setup can be achieved include, butare not limited to: 1) a plethora N of independent calibrated networkinterfaces 201 and 204, and for use by individual students to connect aplethora N of test network interfaces 103, 201 and 204 which are thenconnected to a plethora N of individual IP devices 107, 203 and 206; or2) a single bank of calibrated and test network interfaces 103 with eachstudent's test interface (cable) substituting one of the N calibratedinterfaces (cables), all of which are then connected to an RF combinerand connected to a single cable modem 102 and IP device 107. In thelatter case, careful calibration and choice of cable lengths in both thecalibrated and test network is required and can be performed by thoseskilled in the art. One advantage in particular of multiple cable modemsis that the response of each cable modem to PNM queries can be averagedand interpolated for more accurate determination of the PNM-based craftevaluation response for each student.

FIG. 3 shows what may be perhaps the most typical embodiment of thepresent invention using a remote CMTS 105 and minimal equipment in thetraining facility 101. This setup allows not only a centralized, remote,or distributed architecture to be used for PNM-based craft evaluation,but also leverages the local cable operator's existing network. In somecases, the equipment required in the training facility 101 may belimited to a cable modem 102, some wiring for constructing thecalibrated or test network interface 103, and an IP device 107 forobserving the results of PNM-based craft evaluation. The same process aspreviously described for the calibrated or test network interface 103,cable modem 102 and IP device 107 would be used for PNM-based craftevaluation, the difference is that the fixed calibrated networkinterface 104 is now optional since an actual HFC access network 301 isused as part of the setup, and in this case the CMTS with PNM 105 isowned and operated by the local cable operator, and thus the use of thepresent invention would have to be coordinated with the local cableoperator, and PNM data from the cable operator's CMTS with PNM 105 wouldhave to be made available to the training facility 101. This may be doneeither via a separate cable modem 302 in the facility that feeds aseparate IP device 303, or could be done through an independent packetswitched network 304 such as mobile phone network, DSL network, or fiberto the premises (FTTP) network to an IP device 305 that is eitherlocated external to the training facility 101 or within the facility101. The setup within the training facility 101 may be either the singlestation or the multi-station, or any combination thereof.

It should be noted that this particular embodiment allows the mostrealistic PNM data to be generated from the students' test networkinterface 103, and also minimizes the capital equipment required in thetraining facility 101, which is why it is predicted to be the mosttypical embodiment of the present invention.

It should also be noted that this particular embodiment could beimplemented not just in a training facility, but in fact in a student'shome, community college, local hardware store, and so on, as long ascable modem service was active at that location from the local cableprovider. Additionally, the cable provider could do such testing at anyof their training facilities, or in their headends or hubs where oftencanary cable modems are connected to the network.

FIG. 4 shows the architecture of a stand-alone setup that employsadditional sensors and simulators for characterizing the PNM-based craftevaluation. Since the current method of craft evaluation prior to thepresent invention used primarily visual inspection, the setup in FIG. 4can be used to couple the PNM-based evaluation along with visualinspection, and in particular, take pictures or video of the craft withthe camera 401 and attach that media to the catalog containing thePNM-based craft evaluation data in order to 1) improve the accuracy ofthe PNM-based craft evaluation, 2) develop a detailed catalog of visualclues to specific impairments to enhance the troubleshootingcapabilities of technicians in the field, and 3) use machine learningtechniques on the collected patterns to predict and recommend futurecorrective actions.

In addition, FIG. 4 shows test and measurement equipment 402 that canalso be used to enhance the accuracy, training, and cataloging ofPNM-based craft evaluation. Several examples are described next, butthis list in no way limits the number, type or specific use of suchequipment when used in conjunction with remote or automated PNM-basedcraft evaluation.

In the case of the stand-alone setup in FIG. 1, a noise generator mightbe used to fill the downstream or upstream spectrum with signal so thatfull band RF spectrum captures can be used to identify microreflections,leaks, or impedance mismatches in the cable network. In this case thenoise generator may need the CMTS signal spectrum to be notched out bothon upstream and on downstream so it does not interfere with the PNMcharacterization and operation of the cable modem 102.

It may also be possible to use a cable modem with full band RF spectrumcapture capability in a completely stand-alone manner without need toconnect a CMTS 105 to it, rather using a console connection or othermeans to access the modem's spectrum capture capability without need fora CMTS 105. In this case, the stand-alone setup may be as simple as anoise generator in place of the IP device 106 and CMTS with PNM 105, andthe fixed, calibrated network interface 104 and calibrated or testnetwork interface 103 to perform the PNM-based craft evaluation.

Another way in which test and measurement equipment 402 may be used isas an external noise source to represent ingress that can get into apoorly crafted connector. The equipment 402 could either be a narrowbandtone generator or a broadband noise generator, and the test equipmentsignal could be output to an antenna that is places near the calibratedor test network interface 304. Poor craft skills would result in thisexternal signal ingressing into the cable network and could bedemonstrated via PNM-based technology such as full RF spectrum captureon the downstream or upstream spectrum capture such as will be possiblewith DOCSIS 3.1 PNM technology.

FIG. 5 shows the architecture of a system that allows a plethora M oftraining facilities 501, 501 and 503 to be tied in to a centralizednetwork operations center (NOC) facility 504 and a centralized remoteevaluation function 505. All of the PNM data from each trainingfacility, and in fact the entire network can be collected in the NOC andprovided to the centralized remote evaluation element 506 which may becollocated with the NOC, geographically distinct from it, or acombination of sub-elements in 506 be located in the NOC and othersgeographically distinct from the NOC. In this scenario, results fromdifferent training facilities can be compared and contrasted so that thetechniques and processes used at training facilities that consistentlyperform at the highest craft levels can be used to improve the othertraining facilities.

FIG. 6 shows the details of sub-elements within the remote evaluationelement 506. In one aspect, a cloud-based evaluation function 601 isincluded which allows the results of a plethora of PNM-based craftevaluations to be sorted, classified, cataloged, and prioritized byexisting cloud solutions used by the owners of such a PNM-based craftevaluation system, or the cloud-based evaluation 601 could be replacedwith a crowd-based solution whereby any subject matter experts (SMEs)who have access to the packet switched network 304 and are authorized bythe owners of PNM-based craft evaluation systems to contribute to theoperation of the system can manually perform the functions ofevaluating, sorting, classifying, cataloging, and prioritization of thedata and subsequent evaluations in a PNM-based craft evaluation system.

Alternately, if it is desired to fully automate the evaluation ofPNM-based craft evaluation, the remote evaluation processor 602 can beused, which includes a pattern database 603, a pattern recognitionsystem 604, and an automated evaluator engine 605 that uses results ofthe PNM-based craft evaluation from the plethora of facilities 501, 502and 503 provided by the NOC 504 with machine algorithms apparent tothose skilled in the art to produce the final evaluation of anindividual student's craft to the training facility in which the studentis located. Importantly, this also applies to other types of remotesites such as homes, community colleges, and hardware stores.

FIG. 7 shows the architecture of a stand-alone system that applies theprinciples of the present invention with PNM-based evaluation of craftskills in non-RF network such as fiber optic networks. Examples include,but are not limited to use of a radio frequency over glass (RFoG) systemwithin the overall setup whereby a fiber to the premises (FTTP) system701 based on for example the RFoG standard and architecture is used. TheFTTP system 701 converts both upstream and downstream RF signals intooptical signals that are then passed through a calibratedoptical-to-optical termination network 702 with the result that anoptical calibrated or test network interface 703 (such as testing astudent's ability to do field fusion splicing of FTTP deployment) arealternately connected to an optical networking unit (ONU) 704 and thenceto a cable modem 102 and IP device 107. In this manner, the calibratedand test network interfaces 703 can be compared via PNM-based craftevaluation using the method of the present invention, even though thecraft being tested is actually optical connectors for example, not theRF connectors as previously described.

Other physical media dependent (PMD) systems such as wireless orpowerline media may also be tested in the same manner, substituting theother PMD system for the FTTP and associated optical components justdescribed.

In one exemplary embodiment, there may be a measurement device, such asa cable modem (CM), a cable modem termination system (CMTS), or anotherdevice appropriate for the communications technology, that performs ameasurement. The measurement device takes measurements of a signal usingproactive network maintenance (PNM) parameters and sends them forfurther assessment. The PNM parameters may include, but are not limitedto, spectrum analysis, equalizing coefficients, pre-equalizingcoefficients, signal-to-noise, symbol capture, linear impairments, powerlevels, uncorrectable errors, low RxMER in one or more subcarriers,spectrum that deviates from ideal, pre-equalizer coefficients that havehigh energy off of the main tap, channel estimation coefficients thatvary widely by frequency, a modulation profile indicating subcarriersthat are not transmitting at their expectation or potential, or thelike. In some cases, the communication technology may be Passive OpticalNetwork (PON), with OLTs and ONUs as endpoints. In some cases, theaccess network technology may be Wi-Fi, with access points and networkadapters are endpoints.

The measurement device may measure a variety of characteristics of anupstream or a downstream signal. In some cases, the signal may begenerated for the specific purpose of testing, such as by a noisegenerator. In some cases, the signal may be the result of the normaloperation of a service provided to an end device. In one example, thesignal may be a radio frequency signal. In another example, the signalmay be an optical signal. The measurement device may comprise a memory,a processor, and a communications interface. The memory may storeinstructions that may be executed by the processor in conjunction withthe communications device. The communications interface may assist inmeasuring the PNM parameters, as well as sending results of the PNMmeasurements, and receiving commands/requests from other entities in thenetwork.

In one scenario, after a technician replaces a component of a physicalnetwork, that component needs to be tested to ensure that it has beencorrectly installed (e.g., the performance of the component meets theminimum requirements for normal use). The measurement device may be usedto test the installation of the component (e.g., the performance of thecomponent). The component may be any part of a physical network, such asone or more connectors, cables, patches, devices, and other physicalentities that may be attached to the wireline infrastructure of thephysical network.

The skill of the technician who performed the installation of thecomponent may be directly correlated to the results the performance ofthe component as a result of the installation, or said another way, theskill of the technician may be directly correlated to one or more thePNM measurements of the installed component. In some cases, theinstallation of a component may be for a new configuration of a physicalnetwork. In some cases, the installation of the component may be areplacement of an existing component in need of repair in an alreadyconfigured physical network. Generally, PNM measurements enable theidentification of potential issues related to the component, such asradiofrequency notches, microreflections, roll-off, LTE and radioingress, adjacency, filters, and resonant parking. One or more of theseissues may be a result of the installation, and by association,correlated to the skill of the technician. Accordingly, if one or moreissues are identified, the identified issues may relate to thetechnician's skill.

Once the PNM measurements are taken, a file may be generated with thePNM measurements and other information and sent by the measurementdevice. The file may be associated with a recently installed componentfor a given time or time period, and may include other information aswell (e.g., location, measurement device identifier such as MAC address,etc.). The file may be sent to a remote server that runs an assessmentprogram. The server may part of a network operations center (NOC). Theserver may comprise a memory, a processor, and a communicationsinterface. The instructions for the assessment program may be stored inthe memory and executed by the processor in conjunction with thecommunications interface. The file, amongst other files, may be storedin the memory. In an alternative situation, the assessment program mayrun on the measurement device.

The assessment program may determine an assessment report for thetechnician that represents the technician's skill at installing thecomponent associated with that file (i.e., PNM measurements). Thedetermination of the assessment report may be based on a number offactors, such as the information contained in the received file, anassessment of that information, and other information. In some cases,the assessment report may be based on a comparison of the PNMmeasurements from the file against baseline PNM information (e.g., idealPNM parameters or minimum required parameters). In some cases, thebaseline information may be gathered from averaging a statisticallysignificant number of PNM measurements of successful installations.Alternatively, or in addition to, the baseline information may be basedon a minimum required value for each of the PNM measurements taken. Insome cases, the assessment report may be based on identified issues aspreviously disclosed above. Specific issues may correlate with specificassessment factors of the assessment report. In some cases, theassessment report may be based on additional input, such as additionalsensors or measurements that have been sent to the server from thecomponent/installation location.

The assessment report may comprise of one or more values, descriptions,and/or expert analysis. The assessment report may include a grade, wherea threshold grade indicates a passing or failing assessment for a giventechnician that performed the installation. In some cases, one or moregrades are aggregated and averaged to determine a cumulative assessmentfor a class being taken by the technician. Multiple assessment reportsmay be analyzed to determine trends for a single technician, trackingthe technician's progress from initial craft capabilities to finaltesting and certification of their craft capabilities.

The assessment report may be sent to a display that is proximate to therelevant technician, and/or to an instructor of the relevant technician.In some cases, the assessment report may provide specific feedback tothe technician to correct the installation of the component.

In one scenario, the technician may be part of a group of techniciansthat are part of a class for training technicians to become certified incraft skills for a telecommunications workforce. A classroom may beconfigured with multiple assessment devices, each associated with atechnician of the class such that multiple installations of componentsmay occur and be assessed simultaneously during the class. The servermay receive files (e.g., PNM measurements) in real-time as the class isconducted for each of the technicians that have completed installing acomponent. The server may be configured to send assessment reports ofeach of the technicians in real-time to a display being used, or inproximity to, the respective technicians or teacher of the class.

The server may correlate and catalogue the assessment reports foranalysis (e.g., pattern recognition, averages, comparisons, trends,historical assessments, etc.). The server may perform this analysis ofthe assessment reports once a statistically significant number ofassessment reports are collected and analyzed. The analysis and patternrecognition may correlate other information related to each assessmentreport as well, such as the nature of the installation, the location,the time, and other related factors. The server may perform thisanalysis on the assessment reports for an entire class, from start tofinish, periodically, or as a total for all technicians and eachinstallation performed in the class. This analysis may be used to alterthe class format, requirements, and baselines used to determineassessment reports. The analysis may determine the average, belowaverage, and above average craft capabilities that are possible atdifferent stages of an average technician's training. For example, ahigher assessment report may be expected from a technician in a latertime period of a class as compared to the assessment report expectedfrom that technician at the beginning of the class. This analysis mayalso help the server to dynamically determine a baseline used fordetermining the assessment report for any given point in time for theclass. The analysis may be sent to a display for further review. Theanalysis may prompt an automated change in the instruction of one ormore technicians.

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample only, and not limitation.

What is claimed is:
 1. A system for providing proactive networkmaintenance (PNM) based, substantially real-time feedback relating to atechnician affected network component, comprising: a network connecteddevice in communication with a network, wherein the network connecteddevice comprises: a first processor operatively coupled to memory, and afirst communications interface operatively coupled to the firstprocessor, the first communications interface configured to receive oneor more signals associated with the technician affected networkcomponent the first processor configured to generate proactive networkmaintenance (PNM) data associated with the technician affected networkcomponent based on one or more signals, and the first communicationsinterface configured to send the PNM data; and a server running anevaluator engine in communication with the network connected device,wherein the server comprises: a second processor operatively coupled toa second memory, and a second communications interface operativelycoupled to the second processor, the second communications interfaceconfigured to receive the PNM data associated with the technicianaffected network component from the network connected device, the secondprocessor configured to determine an assessment associated with thetechnician affected network component and an installation technicianbased on the PNM data, and the second communications interfaceconfigured to transmit the assessment to provide real-time feedbackconcerning the installation of the technician affected networkcomponent.
 2. The system of claim 1, wherein the PNM measurementsinclude: power levels; uncorrectable errors; low RxMER in one or moresubcarriers; spectrum that deviates from ideal; pre-equalizercoefficients that have high energy off of the main tap; channelestimation coefficients that vary widely by frequency; and a modulationprofile indicating subcarriers that are not transmitting at theirexpectation or potential.
 3. The system of claim 1, wherein the signalis a radio frequency transmission.
 4. The system of claim 1, wherein thesignal is an optical transmission.
 5. The system of claim 1, wherein theserver is at a remote location relative to the network connected device,and wherein the network connected device is a cable modem.
 6. The systemof claim 1, wherein the technician affected network component is a cableor a connector between one or more cables of the network.
 7. The systemof claim 1, wherein the signal is generated for training theinstallation technician.
 8. The system of claim 1, wherein the signal isgenerated for normal operation of the network.
 9. The system of claim 1,wherein the network is a training network not connected to a local cableoperator.
 10. The system of claim 1, wherein the network is a localcable operator's network.
 11. The system of claim 1, wherein the secondcommunications interface is further configured to send an assessmentreport to a display of a teacher of the installation technician in asecond facility different than a first facility of the measurementdevice.
 12. The system of claim 1, further comprising a noise generator,the noise generator configured to generate a noise signal to simulateexternal noise ingressing into the network at the technician affectednetwork component, wherein the PNM measurements are based in part on howmuch of the noise signal is captured.
 13. The system of claim 1, whereinan assessment report is based in part in comparing calibration PNMmeasurements taken prior to the installation of the technician affectednetwork component.
 14. The system of claim 1, wherein an assessmentreport includes key performance indicators and a grade of skills of theinstallation technician, wherein the PNM measurements of the technicianaffected network component correlate with the key performance indicatorsand the grade.
 15. The system of claim 1, wherein the PNM measurementsare sent in a file, and the file includes a location where the PNMmeasurements are taken, and wherein an assessment report is furtherbased on additional information known about the location.